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ABSTRACT 

This paper discusses the flight test results of a real-time Digital Elevation Model (DEM) integrity monitor for Civil Aviation 
applications. Providing pilots with Synthetic Vision (SV) displays containing terrain information has the potential to 
improve flight safety by improving situational awareness and thereby reducing the likelihood of Controlled Flight Into 
Terrain (CFIT). Utilization of DEMs, such as the digital terrain elevation data (DTED), requires a DEM integrity check and 
timely integrity alerts to the pilots when used for flight-critical terrain-displays, otherwise the DEM may provide hazardous 
misleading terrain information. The discussed integrity monitor checks the consistency between a terrain elevation profile 
synthesized from sensor information, and the profile given in the DEM. The synthesized profile is derived from DGPS and 
radar altimeter measurements. DEMs of various spatial resolutions are used to illustrate the dependency of the integrity 
monitor's performance on the DEMs spatial resolution. The paper will give a description of proposed integrity algorithms, 
the flight test setup, and the results of a flight test performed at the Ohio University airport and in the vicinity of Asheville, 
NC. 
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1. INTRODUCTION 

Synthetic Vision Systems (SVS) are intended to provide pilots with an advanced display that depicts terrain information as 
well as other information about the external environment such as obstacles and traffic. NASA's aviation safety program is 
investigating SVS as an enabling technology for a wide variety of applications such as reduction of Controlled Flight Into 
Terrain (CFIT), low-visibility surface operations, advanced precision approach procedures, and low-visibility loss-of-control 
scenarios. This range of applications is discussed in NASA's “Concept of Operations for Commercial and Business Aircraft 
Synthetic Vision Systems” [1], The focus in this paper is the use of SVS for CFIT-reduction. To mitigate CFIT, several 
strategies have been, and are being, pursued by the government and private sectors, such as Terrain Awareness and Warning 
Systems (TAWS). TAWS is currently being mandated by the FAA for use on all turbine-powered U.S. -registered airplanes 
type certified to have six or more passenger seats [2], It is important to note that TAWS is purely an advisory system. 

In an SVS, CFIT reduction is achieved by increasing pilot awareness of nearby terrain by displaying it on either a head up 
display (HUD) or head down display (HDD). Especially in flight-critical SVS applications, display of misleading terrain 
information must be avoided. The ad-hoc integration of terrain elevation data into the cockpit may conceivably create 
scenarios that lead to accidents because of the high level of realism provided by SVS displays. The compelling nature of an 
SVS underlines the mandate for integrity assurance. 

This paper describes the implementation of a real-time DEM integrity monitor in order to reduce the probability of an 
undetected DEM error and thus reduce the potential for presenting hazardous misleading terrain information. Sensor 
information from DGPS and radar altimeter are used to generate a synthesized, or “sensed”, terrain elevation profile. This 
profile is compared to the terrain elevation profile derived from the DEM and if there are statistically significant 
inconsistencies between the two, an integrity alarm will be generated and presented to the pilot in some fashion (to be 
determined). A prototype SVS DEM integrity monitor was developed based on the availability of radar altimeter and 
augmented GPS. The augmented GPS was provided by a prototype of the Local Area Augmentation System (LAAS). The 
prototype integrity monitor was installed on Ohio University's DC-3 and flight test data was collected at two locations: the 
vicinity of the Ohio University airport (UNI) and the vicinity of the Asheville, NC, airport (AVL). Various test statistics 
were compared for a number of flight segments flown at UNI and AVL. To evaluate the dependency of the integrity monitor 
performance on the DEM resolution and accuracy, the test statistics were computed for DTED level 0 and level 1 DEMs. 

Techniques similar to the one proposed for terrain database integrity monitoring have been used for terrain-based navigation. 
Examples are Terrain Contour Matching (TERCOM) [3,4], the Sandia Inertial Terrain-aided Navigation (SITAN) [3,4], and 



Terrain Profile Matching, TERPROM [5]. These systems utilize a radar altimeter combined with another positioning source 
to derive synthesized terrain elevations. The Autonomous Precision Approach and Landing System (APALS) utilizes a 
modified weather radar in combination with GPS and an Inertial Measurement Unit (IMU) to enable precision approach. It is 
important to note that these systems were designed to enable position “fixing” in support of autonomous navigation, not as 
integrity monitors for a display suite such as SYS. 


2. SYNTHETIC VISION SYSTEM PERFORMANCE 

Figure 1 shows a conceptual block diagram of an SVS. The strategic display, the tactical display, and the HUD are the visual 
components of the SVS and form the interface between the SVS processor and the pilot [6]. The SVS processor takes the 
required inputs to generate the information that is being depicted on the displays. Furthermore, the SVS processor is 
connected to a database server: an onboard unit that stores and manages the data that supports the SVS [7]. This paper 
concentrates on the terrain data that is being stored and managed in this server. Digitally stored terrain elevation data is 
referred to as a Digital Elevation Model or DEM. A DEM integrity function would be located within this database server and 
would require database server inputs such as external sensor information for independent verification of the DEM in real- 
time. 
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Figure 1. Conceptual SVS system 


Subsequent to the identification of the SVS target operations, the development of operational requirements, and the 
functional analysis, SVS system performance requirements must be developed. SVS system performance can be specified in 
terms of a set of system performance parameters: accuracy, integrity, availability, and continuity-of-service. This paper will 
focus strictly on the integrity parameters of the terrain database. SVS-required integrity is dependent on how the SVS is 
being used; Non-essential use (e.g. clear-day, long-term flight planning); Strategic essential use (e.g. near-term flight 
planning in low visibility); or Tactical critical use (e.g. low visibility navigation near airports, autopilot). 


3. DATABASE INTEGRITY MONITORING 

Integrity monitoring of DEM information is accomplished by a consistency check between the information stored in the 
databases and the information derived from external sensors. Based on the sensor technology that is being used to monitor 
the DEM integrity, three categories of integrity monitors can be considered: (1) integrity monitors based on downward 
looking (DWL) sensor information; (2) integrity monitors based on forward looking (FWL) sensor information; and (3) 
integrity monitors based on both DWL and FWL sensor information (see Figure 2). 

In the proposed DWL integrity scheme, sensor information from a Differential Global Positioning System (DGPS) and radar 
altimeter are used to generate a synthesized, or “sensed”, terrain elevation profile. This profile is compared to the terrain 
elevation profile stored in the database server and if there are statistically significant inconsistencies between the two profiles. 






an integrity alarm will be generated and presented to the pilot in some fashion. The difference between the synthesized and 
stored profiles is referred to as the absolute disparity, whereas the difference between two successive absolute disparities is 
referred as a successive disparity. Based on the underlying and/or specified error characteristics of the sensors and the 
DEMs, probability density functions can be derived for the absolute and successive disparities. Test statistics such as the 
mean square difference, the mean absolute difference, and the cross-correlation, are derived from these metrics and 
statistically assessed to obtain integrity thresholds and minimum detectable biases. 

Integrity algorithms based on FWL sensors would statistically assess the correlation (in time and space) between terrain 
features observed by FWL sensors and the terrain features derived from the DEM. This paper does not discuss these 
techniques. Examples of FWL sensors that may be utilized for such an integrity monitor are weather radars, millimeter wave 
radars, or modified (forward -looking) radar altimeters. 

The metrics used to express the degree of agreement or consistency between the synthesized and DEM profiles in a DWL 
sensor integrity scheme, are the absolute and successive disparities [7]. In this paper the discussion is limited to the use of 
absolute disparity because of the insensitivity of the successive disparities to biases and low-frequency errors [13]. The 
absolute disparity is given by: 


P(tj ) — h SYNT ( tj ) h DTED (f ; ) (1) 

where h SYNT is the synthesized height and h DTED is the height as derived from the DEM. Both elevations are defined at time 
tf 



Figure 2. Integrity monitors based on downward- and forward-looking sensors 


Ideally, the difference between the stored and synthesized elevation would be zero. However, nominal errors in the sensor 
performance and stored entries in the DEM cause the absolute disparity to have a nominal error distribution [8]. 
Overbounding the bias and random error components of both the sensor and the DEM yields a probability density function 
(PDF) for the absolute disparity under the fault -free condition. This leads to the following null hypothe sis H 0 : 

H 0 : p ~ N(0,o 2 p ) (2) 

where N(0, T> p 2 ) is a normal distribution with a standard deviation of <f> p . The standard deviation <t> p can be derived from the 
individual standard deviations of the sensor error PDFs and specified error characteri Stic of the DEM. 

When a fault in the form of a bias -error occurs the PDF of the absolute disparity will contain a bias component also. Hence, 
the following expression for an alternative hypothesis Hf. 


Hi :p~N(\i B ,c 2 p ) 


(3) 


where : B . is the bias-error or fault. Alternative hypotheses can be generated based on other expected error classes or faults. 
However, this discussion will be restricted to the analysis of bias -like errors (or faults). 

For the implementation of an integrity monitor a t est statistic must be derived. Examples are the Mean Absolute Difference 
(MAD) and Mean Square Difference (MSD). The MSD slightly outperforms the MAD [9] and is therefore chosen as the test 
statistic, T, for proposed integrity monitor: 


N 1 N 

T = —MSD AD =—Yp 2 (t i ) (4) 

a p g p i=i 

Under H 0 , the PDF of T is found to be a chi-square distribution with N degrees of freedom. Under II /, the PDF of T is found 
to be a non-central chi -square distribution with N degrees of freedom and non-centrality parameter 8 . The non -centrality 
parameter is related to the bias -error or fault as follows [10]: 


X = 



(5) 


SVS system requirements must specify a probability of detection under fault free conditions, Pffd, a probability of missed 
detection, P M d, and the integration time N. Fault -free conditions are those conditions in which errors behave only as expected 
(or as specified for either the sensor or the DEM). Given the value for P ffd, a threshold can be calculated from H 0 . This is 
illustrated by the solid area underneath the curve in Figure 3. P F fd can also be thought of as the probability of false alarm. 

For the real-time integrity monitor used during the flight -tests, an integration time of 50 seconds (N= 50) and a P F fd=10" 4 were 
chosen. These parameter values result in a threshold value of T D = 96. The probability of missed detection, P md, or a 
continuation of operation under the alternative hypothesis, results in a minimum detectable bias (MDB): that bias for which 
the probability of a missed detection is equal to P M d- The MDB can be derived from T D and P M d by identifying that non- 
central chi -squared distribution for which the gray area in Figure 3 equals P M d- The MDB can then be obtained from the non - 
centrality parameter in equation 5. For example, f or Pmd= 10" 7 a MDB value of 33.96 meters results. Note that this value is 
quite significant. 



A 

H 0 


\ 

/ 

H, 

p 

1 f*MD \ 

r( 


1 xy 



t d T 


Figure 3. Ho and /// hypotheses for the mean square difference test statistic. 


Note that lower values of P M d result in lower values of the MDB [11]. Furthermor e, a shorter integration time results in 
higher values for the MDB. 





4. TERRAIN DATABASE ELEVATION MEASUREMENTS 


Elevation databases stored digitally and defined at discrete spatial points are often referred to as digital elevation models or 
DEMs. A variety of sources provide DEMs specified by a number of parameters, such as the postspacing or spatial 
resolution, the horizontal and vertical references or datums, and the circular and linear errors. The circular error probability 
(CEP) represents the horizontal accuracy specification on the post-position, whereas the linear error probability (LEP) or 
vertical error specifies the accuracy in the vertical direction (height). Various DEMs were available for the Asheville, NC 
and Athens, OH areas to support the flight -test analyses and real-time implementation. The DEMs used in this paper are the 
Digital Elevation Terrain Data (DTED) levels 0 and 1 provided by the National Imaging & Mapping Agency (NIMA). For 
the geographic areas of interest both databases have a similar horizontal and vertical accuracy specification: CEP < 50m, 
90% (<I> hor = 30.4 m), LEP < 30m, 90% ( 4> ver = 18.2 m). The major difference between DTED Level 0 and 1 lies in their 
spatial resolutions; whereas DTED level 1 has a post -spacing of 3 arc-seconds. DTED level 0 has a post-spacing of 30 arc- 
seconds. A limited spatial resolution may result in significant interpolation errors because the terrain is truly under -sampled. 
In geographic areas with a large terrain standard deviation (“rough” ter rain) [9], large interpolation errors may be expected! 
However, it is in these geographic areas that misleading terrain information is most likely to be hazardous. The effects of the 
limited resolution and their impact on the nominal performance will be illustrated in the flight test result section of this paper. 


5. SYNTHESIZED ELEVATION MEASUREMENTS 

The synthesized elevation measurements are obtained by subtracting the height above ground level measured by the radar 
altimeter from the altitude above mean sea level computed by the Differential Global Positioning System (DGPS). Examples 
of DGPS that may be utilized in the real -time integrity monitor implementation are the Wide Area Augmentation System 
(WAAS) [12] or Local Area Augmentation Systems (LAAS) [12]. In this paper, high -precision Kinematic GPS was used for 
post-processing purposes. The radar altimeter measures the height above ground level ( AGL) of the radar altimeter receiving 
antenna (mounted on the bottom of the aircraft). However, it is important to realize that the radar altimeter measurement 
accuracy is a function of the altitude and the rate at which the altitude changes. Furthermore, banking of the aircraft may 
cause the radar altimeter beam to measure a “slant -range” instead of a “plumb bob” range [13]. 

Given the accuracies specified in [9,1 1], values for <t> p can be computed. Table 1 gives an overview of the values for various 
combinations of DGPS and DEMs. Note that this table includes <t> p values obtained when using DTED as well as th e Shuttle 
Radar Topography Mission (SRTM) -derived elevation database. The SRTM is a joint project of NIMA and NASA to record 
radar data that will be used to generate of a digital topographic map of 80% of the Earth's land surface [14]. 
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Table 1. Standard deviation of the absolute disparity under fault -free conditions. 


6. FLIGHT TEST SETUP AND RESULTS 

Data from two flight tests was used to evaluate the integrity monitor's behavior as a function of various DEMs and sensor 
packages; one in the vicinity of Asheville, NC and one in the vicinity of Ohio University in Athens, OH. In b oth cases Ohio 
University's real-time synthetic vision prototype was also tested. Figure 4 shows a block diagram of the prototype hardware. 
A Commercial-Off-The -Shelf (COTS) Honeywell Inertial Navigation System (INS) provides the aircraft attitude, headi ng, 
and airspeed information. The aircraft LAAS unit relays the airborne position and altitude above mean sea level to the 
Central Processing Unit (CPU). The LAAS differential corrections are received from a prototype LAAS ground station that 
makes use of spread-spectrum radios as short-range low -power datalink. A COTS Honeywell radar altimeter unit is 
connected to the CPU via a Microcontroller interface unit. The prototype provides A/C state and terrain information 
graphically to the pilot on a flat pa nel liquid crystal display that functions as HDD. The HDD software was developed by 
Delft University of Technology and is used through a memorandum of agreement. 
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Figure 4. SVS prototype hardware 



Figure 5. Software flowchart for SVS prototype 

The CPU runs the real-time Integrity Monitor software under the QNX real -time operating system (RTOS). A high -level 
flowchart of the software is shown in Figure 5. DGPS, radar altimeter, and DTED data are retrieved by their respective 
handlers and sent to the parity handler. The parity handler computes the absolute disparities and test statistic. Based on the 
computed test statistic and the integrity threshold T D , which has been computed a priori, the parity -handler determines if there 
is an integrity viola tion and sets an output flag accordingly. The display handler generates the information required by the 
HDD and sends it via a TCP/IP connection to the dedicated HDD computer. Synchronization of the messages is required to 
enable a fair consistency check among measurements taken at approximately the same reference time. This function is 
implemented by the time-handler. 


6.1 Ohio University airport (UNI) flight test 


The first set of data was collected during a flight test with Ohio University’s DC -3 in the vicinity of Ohio University airport 
(UNI) in August 2000. The data shown here was derived in post -flight to enable a direct performance comparison between 
the integrity monitor’s performance with LAAS / KGPS and DTED level 0 / DTED level 1. Figure 6 sho ws the ground- 
tracks for the 14 approaches (2 to runway number 7, 12 to runway number 25). Note the curvature of the approaches to 
runway number 7. The resultant bank angles introduce errors due to the measurement mechanism of the radar altimeter [13]. 


Runway 7 Approach Runway 25 Approach 



Figure 6. Approaches to UNI runway 7 and 25 during the August 2000 flight test. 



Figure 7. T -statistic and threshold for the approach to UNI runway 7 utilizing DTED level 0 and 1 


The T -statistic results for the approaches towards runway 7 and 25 are shown in Figures 7 and 8, respectively. The graph on 
the left shows the monitor’s behavior when utilizing DTED level 0, whereas the graph on the right shows the monitor’s 
behavior when utilizing DTED Level 1. One can observe an obvious difference in int egrity monitor performance: using 
DTED level 0 causes the integrity threshold to be exceeded. In other words the interpolation error introduced by the large 
spatial resolution of DTED level 0 causes the actual fault -free error characteristic to deviate f rom the specified fault-free error 
characteristic. This will then result in the occurrence of false alarms. It is important to remember that the horizontal and 
vertical DEM accuracies are defined for one point in space, the post -space, not for the interpolated position. 



Figure 8. T -statistic and threshold for the approach to UNI runway 25 utilizing DTED level 0 and 1. 


Furthermore, it can be seen in figure 7 that the test statistics during the approaches to runway 7 are much larger than the test 
statistics during the approaches to runway 25. The reason for this is error is twofold: first the terrain underneath the 
approach to runway 7 has a higher terrain standard deviation than the terrain underneath the approach to runway 25 and 
secondly, the large bank angles cause a mismatch between the radar altimeter measurements and the interpolated DEM 
elevation [13]. 

6.2 Asheville, NC (AVL) 

Flight tests were performed in the vicinity of the Asheville, NC, airport (AVL) in September 2000. Figure 9 shows the 
ground tracks for these approaches. Asheville, NC was chosen for its relative high terrain standard deviation and the 
availability of multiple alternative terrain databases including photogrammetry data. In 1999, NASA flight tested their SVS 
prototype cn the Total In -Flight Simulator (TTFS) at this same airport. 



Figure 9. Ground -tracks for the approaches to runway 16 and runway 34 at AVL 


The proposed test statistics were calculated for the 14 flight segments flown. The set of flight segments consist ed of 
Instrument Landing System (ILS) approaches to runway 16 and 34.The test statistics for these approaches are given in figures 
10 and 11, respectively. The ILS approach to runway 34 has a different terrain profile than the approach to runway 16. 
During the initial approach to runway 34, the terrain is characterized by large variations, but during final approach the terrain 
variations become significantly smaller. During the approach to runway 16, the frequency of undulations in the terrain 
remains significant until the aircraft reaches the runway. Again, when using DTED level 1, no integrity violations were 
detected. However, when using DTED level 0, the thresholds are exceeded for approaches to both runways. Again, 


interpolation errors due to a li mited spatial resolution are the cause of these false alarms. In [11,13] the AVL DTED segment 
was compared to high-accuracy photogrammetry data and no faults were identified. 



Figure 10. T -statistic and threshold for the approach to AVL runway 16 util izing DTED level 0 and 1. 
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Figure 11. T -statistic and threshold for the approach to AVL runway 34 utilizing DTED level 0 and 1. 


The data shown in figures 10 and 11 was generated using LAAS positions and altitudes. A very similar integrity monitor 
performance is found when using post-flight KPGS. This is to be expected because the standard deviation of the PDF of the 
absolute disparity is dominated by the vertical standard deviation of the DEM. 

7. SUMMARY AND CONCLUSIONS 

A prototype SVS with real-time DEM integrity monitor was flight -tested at UNI and AVL. The test statistic values for both 
geographic areas were consistent and below the threshold for all approaches using DTED level 1. Use of DTED level 0 in 
the real-time integrity monitor caused the integrity threshold to be exceeded on various occasions. Each one of these cases 
corresponded to flight over terrain with a large terrain standard deviation. It is concluded that the limited spatial resolution of 
DTED level 0 causes the PDF that underlies t he test statistic to be off-nominal, resulting in false alarms and possibly missed 
detections. DTED level 1 was shown to be sufficient. However, degraded performance of DGPS or radar altimeter may 
necessitate even higher quality terrain databases. 

Comparing the results in this paper with the results in [13] indicate a degraded performance. This can be attributed to the use 
of different radar altimeters in both test aircraft. Additional flight -tests using a variety of radar altimeters will be necessary to 
validate the general performance of the DEM integrity monitor. 


Even though the integrity monitor performance looks promising, it must be determined during the system requirements 
development if the resulting MDBs are acceptable. Higher accuracy data bases such as the SRTM, longer integration times, 
or augmentation with FWL integrity schemes may be necessary to guarantee DEM integrity during all operational scenarios. 
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